Control device and control method

ABSTRACT

The present technology relates to a control device and a control method capable of providing a more convenient electroencephalogram input user interface.Provided is a control device including a detection unit configured to perform detection of a brain wave included in a measured biometric signal of a user and detection of a user action based on information other than the brain wave included in the biometric signal, and a processing unit configured to perform a predetermined process based on the brain wave in a case where the user action is a predetermined action. For example, the present technology can be applied to a measurement device capable of measuring a brain wave signal.

TECHNICAL FIELD

The present technology relates to a control device and a control method,and moAre particularly to a control device and a control method capableof providing a more convenient electroencephalogram input userinterface.

BACKGROUND ART

In recent years, research and development for analyzing brain waves andapplying them to interfaces have been actively conducted.

For example, Patent Document 1 discloses a safe drive assistance systemfor determining whether a predetermined electroencephalogram pattern isdetected in a brain wave measured by an electroencephalogram measurementunit in a case where it is determined whether a driver is in a drivablestate by comparing allowable information about the driver with biometricinformation.

CITATION LIST Patent Document

Patent Document 1: Japanese Patent Application Laid-Open No. 2019-199177

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

However, the electroencephalogram measurement has a large artifact dueto the head myoelectric potential of the user, and the usage scene iscurrently limited. Therefore, a more convenient electroencephalograminput user interface has been required to be proposed.

The present technology has been made in view of such a situation, and anobject thereof is to provide a more convenient electroencephalograminput user interface.

Solution to Problems

A control device according to an aspect of the present technology is acontrol device including a detection unit configured to performdetection of a brain wave included in a measured biometric signal of auser and detection of a user action based on information other than thebrain wave included in the biometric signal, and a processing unitconfigured to perform a predetermined process based on the brain wave ina case where the user action is a predetermined action.

A control method according to an aspect of the present technology is acontrol method including a control device detecting a brain waveincluded in a measured biometric signal of a user and detecting a useraction based on information other than the brain wave included in thebiometric signal, and performing a predetermined process based on thebrain wave in a case where the user action is a predetermined action.

In a control device and a control method according to an aspect of thepresent technology, a brain wave included in a measured biometric signalof a user is detected, a user action based on information other than thebrain wave included in the biometric signal is detected, and apredetermined process based on the brain wave is performed in a casewhere the user action is a predetermined action.

The control device according to an aspect of the present technology maybe an independent device or an internal block constituting one device.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating a registration example of a triggeraction and a pass-thought.

FIG. 2 is a diagram illustrating a registration example of a triggeraction and a pass-thought.

FIG. 3 is a diagram illustrating an example of a variation of a triggeraction.

FIG. 4 is a diagram illustrating an example of a pattern of a voiceinstruction.

FIG. 5 is a diagram illustrating an authentication example ofelectroencephalogram personal authentication.

FIG. 6 is a diagram illustrating an authentication example ofelectroencephalogram personal authentication.

FIG. 7 is a diagram illustrating a registration example ofelectroencephalogram personal authentication using an auditory inductionresponse.

FIG. 8 is a diagram illustrating an authentication example ofelectroencephalogram personal authentication using an auditory inductionresponse.

FIG. 9 is a diagram illustrating a registration example ofelectroencephalogram personal authentication using a visual inductionresponse.

FIG. 10 is a diagram illustrating an authentication example ofelectroencephalogram personal authentication using a visual inductionresponse.

FIG. 11 is a diagram illustrating a first example of recall search.

FIG. 12 is a diagram illustrating a second example of recall search.

FIG. 13 is a block diagram illustrating a configuration example of anelectroencephalogram system to which the present technology is applied.

FIG. 14 is a view illustrating an arrangement example of an electrodeand a reference electrode provided in an earphone.

FIG. 15 is a view illustrating an arrangement example of an electrodeand a reference electrode provided in an HMD.

FIG. 16 is a diagram illustrating a configuration example of a tablerecorded in a signal recording unit.

FIG. 17 is a diagram illustrating an example of a trigger action signalrecording table.

FIG. 18 is a diagram illustrating an example of a triggeraction-specific electroencephalogram application table.

FIG. 19 is a diagram illustrating an example of a brain wave signalrecording table.

FIG. 20 is a block diagram illustrating another configuration example ofthe electroencephalogram system to which the present technology isapplied.

FIG. 21 is a block diagram illustrating another configuration example ofthe electroencephalogram system to which the present technology isapplied.

FIG. 22 is a block diagram illustrating another configuration example ofthe electroencephalogram system to which the present technology isapplied.

FIG. 23 is a flowchart illustrating a flow of a registration process ofa trigger action and a pass-thought.

FIG. 24 is a flowchart illustrating a flow of a registration process ofa trigger action and a pass-thought.

FIG. 25 is a flowchart illustrating a flow of an electroencephalogrampersonal authentication process.

FIG. 26 is a diagram illustrating an authentication registration exampleof personal authentication using auditory induction.

FIG. 27 is a diagram illustrating an authentication registration exampleof personal authentication using auditory induction.

FIG. 28 is a diagram illustrating an authentication registration exampleof personal authentication using visual induction.

FIG. 29 is a diagram illustrating an authentication registration exampleof personal authentication using visual induction.

FIG. 30 is a diagram illustrating an application example of a musicdistribution service.

FIG. 31 is a diagram illustrating an application example of a gamescreen.

FIG. 32 is a diagram illustrating an application example of a loadingscreen.

FIG. 33 is a diagram illustrating an application example of a userselection screen.

FIG. 34 is a diagram illustrating an application example of a userselection screen.

FIG. 35 is a block diagram illustrating a configuration example of abrain information system to which the present technology is applied.

FIG. 36 is a block diagram illustrating another configuration example ofa brain information system to which the present technology is applied.

FIG. 37 is a block diagram illustrating a configuration example of anfNIRS sensor.

FIG. 38 is a diagram illustrating an arrangement example of fNIRSsensors provided in an earphone.

FIG. 39 is a diagram illustrating an arrangement example of fNIRSsensors provided in an HMD.

FIG. 40 is a diagram illustrating a configuration example of a tablerecorded in a signal recording unit.

FIG. 41 is a diagram illustrating an example of a CBPA table.

FIG. 42 is a flowchart illustrating a flow of an authenticationregistration process.

FIG. 43 is a flowchart illustrating a flow of an authenticationregistration process.

FIG. 44 is a block diagram illustrating a configuration example of acomputer.

MODE FOR CARRYING OUT THE INVENTION 1. First Embodiment

Currently, a multi-factor authentication system using a plurality ofverification elements is often used for authentication in which highsecurity is required, such as in a bank. In order to performmulti-factor authentication, a user who intends to receiveauthentication is required to present two or more categories ofauthentication elements among the following (a) to (c).

(a) Knowledge: information known only by the user (for example, apassword)

(b) Belongings: a thing that only the user has (for example, ATM card)

(c) Biometric information: a feature (for example, fingerprint) thatonly the user has

However, the multi-factor authentication system is systemicallycomplicated and expensive. In addition, from the viewpoint of userexperience (UX), since there are many steps, it is hard to say that itis user-friendly.

Brain waves that can be read from electroencephalograph (EEG), and brainactivities that can be read from magnetic resonance imaging (MRI) andnear-infrared spectroscopy (NIRS) have features indicating individualdifferences.

When a pass-thought known only by the user can be read with accuracywith which it is possible to discriminate between individuals,authentication can be performed in one process by using knowledge knownonly by the user corresponding to the above-described (a) as a firstauthentication element and biometric information such as a brain featureamount of the user corresponding to the above-described (c) as a secondauthentication element at the same time.

The pass-thought is brain activity information specific to a case wherean arbitrary object is perceived and recalled. The arbitrary objectincludes a voice, an image, an odor, a tactile sense, a taste sense, andthe like. These may be actually recognized and perceived, or may berecalled or imagined although not actually perceived.

The recalling or imagining may be performed on the basis of a direct orspecific instruction indicating the object, or may be performed on thebasis of an abstract instruction. As an example of the former case, acase where the user imagines an apple on the basis of an instruction toimagine an apple is assumed. In addition, as an example of the latter, acase where the user imagines an apple on the basis of an instruction toimagine a favorite fruit is assumed. The brain activity information isinformation obtained by brain wave, MRI, or the like, and may be definedas a measurement signal, or a feature amount extracted from themeasurement signal, or a space including the feature amount.

On the other hand, the electroencephalogram measurement has a largeartifact due to the head myoelectric potential of the user, and theusage scene is currently limited.

That is, at the time of electroencephalogram measurement, an artifactcaused by blinking or myoelectric potential of the chin is a problem.Therefore, in normal electroencephalogram measurement, measurement withthe eyes closed is performed in a resting state. For this reason, thereis a problem that a user interface (UI) using a brain wave signal as aninput signal has a limited use scene and is difficult to implement insociety.

Therefore, in the present technology, by providing a user interface ofan electroencephalogram measurement machine using a head myoelectricpotential, processing regarding a brain wave such aselectroencephalogram personal authentication having a user interface(UI) that is natural and easy for all people to understand is realizedwhile solving these problems. Hereinafter, a first embodiment will bedescribed with reference to the drawings.

First Example

FIGS. 1 and 2 illustrate a registration example of a trigger action anda pass-thought used in electroencephalogram personal authentication.

In FIGS. 1 and 2 , the user holds a mobile terminal 10A such as asmartphone in the hand, and wears on the ear an earphone 20A connectedto the mobile terminal 10A in a wireless or wired manner. Althoughdetails will be described later, the earphone 20A is provided with aplurality of electrodes, and can measure a biometric signal such as amyoelectric signal or a brain wave signal from the head of the user.

The user operates the menu screen of the electroencephalogram personalauthentication application being activated on the mobile terminal 10A toselect registration of a trigger action and a pass-thought (S11). Theelectroencephalogram personal authentication application is anapplication that performs personal authentication using a brain wavemeasured from the head of the user, and is an example of theelectroencephalogram application. The trigger action is an action (apredetermined action or the like of the user) serving as a trigger whenthe predetermined process based on the brain wave is performed.

In response to this selection operation, the mobile terminal 10Apresents a user interface that prompts the user to take a predeterminedaction with an image, text, sound, or the like (S12). In this example, ascreen including a message prompting to close eyes and start an image ofa pass-thought according to a voice instruction is displayed by an imageand text. The voice instruction is an instruction by voice and is anexample of an instruction that is an explicit instruction from thesystem.

When the user closes the eyes according to the message displayed onmobile terminal 10A, the myoelectric signal corresponding to the actionis measured by the electrode of the earphone 20A (S13). In this example,in response to the user's eye closing, the myoelectric signal includingthe waveform W11 of FIG. 1 is measured and recorded as a trigger actionsignal.

Thereafter, a voice instruction that prompts the user with the eyesclosed to image a pass-thought to be used as a key forelectroencephalogram personal authentication is output from the earphone20A (S14).

When the user imagines a pass-thought according to the voiceinstruction, the brain wave signal is measured by the electrode of theearphone 20A (S15). In this example, when the user images a cat, a brainwave signal including the waveform W12 in FIG. 2 is measured andrecorded in association with an electroencephalogram personalauthentication application as an electroencephalogram application andeye closing as a trigger action.

When the trigger action and the pass-thought are registered, a voicenotifying the user that the registration is completed and that the eyesmay be opened is output from the earphone 20A (S16). In addition, a userinterface indicating that the registration is completed is presented inthe mobile terminal 10A by an image, text, sound, or the like (S17). Inthis example, a screen including a message indicating completion ofregistration is displayed by an image and a text.

In this manner, the user performs an action or the like serving as atrigger action according to the user interface (UI) such as a voiceinstruction or images a thing or the like used as a pass-thought,whereby the trigger action and the pass-thought used in theelectroencephalogram personal authentication are registered.

In the above description, the example in which the user registers theeye closing action as the trigger action (S12 and S13 in FIG. 1 ) isdescribed. However, as illustrated in FIG. 3 , another action by theuser may be registered as the trigger action by presenting a userinterface that prompts an action such as blinking twice, closing theeyes with the neck down, or closing the eyes and moving the mouth (jaw)only four times at predetermined intervals (S12′).

As a result, according to the presented user interface, the userperforms an action such as blinking twice, closing the eyes with theneck down, or moving the mouth (jaw) four times after closing the eyes,whereby the myoelectric signal including the waveform W11′ correspondingto the action is measured and recorded as the trigger action signal(S13′). That is, the user interface presented here is an instructionthat is an explicit instruction from the system, and prompts the user toperform a trigger action.

In addition, in the above description, the example in which the userimages an arbitrary object (thing or the like) as the pass-thought (S14and S15 in FIG. 2 ) is described. However, as illustrated in FIG. 4 , itis not limited to a voice instruction that prompts the user tocompletely freely imagine a favorite image, but the image range of thepass-thought may be restricted by a voice instruction such as “pleaseimagine an animal you like”, and the degree of freedom may be lowered.Alternatively, a person, a scene, a motion of the body of the user, aword, a number, a melody, a song, or the like may be imagined, or aselection form such as a secret question may be used.

As a result, a brain wave signal including a waveform W12′ correspondingto the image of the answer of the user to the more limited favoriteanimal or secret question is measured and recorded in association withthe electroencephalogram application and the trigger action (S15′).

Note that, at the time of registering the trigger action and thepass-thought described above, by repeatedly performing the processes ofsteps S12 to S17 in FIGS. 1 and 2 a plurality of times, it is possibleto register the trigger action signal and the brain wave signal withhigher accuracy, and it is possible to increase the accuracy of theelectroencephalogram personal authentication using the trigger actionand the pass-thought.

FIGS. 5 and 6 illustrate an authentication example ofelectroencephalogram personal authentication (pass-thoughtauthentication) using a trigger action and a pass-thought.

In FIGS. 5 and 6 , the user holds the mobile terminal 10A such as asmartphone in the hand, and wears on the ear the earphone 20A connectedto the mobile terminal 10A in a wireless or wired manner. The earphone20A is operating in a trigger action measurement mode (S21).

At this time, when the user closes the eyes, the myoelectric signalcorresponding to the action is measured by the electrode of the earphone20A (S22). In this example, the myoelectric signal including a waveformW21 of FIG. 5 is detected as the trigger action signal, and the triggeraction recognition process is initiated in the earphone 20A.

In this trigger action recognition process, it is checked whether atrigger action signal having a similarity within a predeterminedthreshold value range exists in the trigger action signal recorded atthe time of registration. In addition, from the relationship between therecorded trigger action signal and the electroencephalogram application,notification of the detection of the trigger action signal is providedto the corresponding electroencephalogram application.

In this example, since the waveform W21 of the myoelectric signaldetected as the trigger action signal at the time of authentication hasa similarity within the predetermined threshold value range with thewaveform W11 of the myoelectric signal recorded as the trigger actionsignal at the time of registration, the trigger action is eye closing,and the corresponding electroencephalogram application is theelectroencephalogram personal authentication application (S23).

When the trigger action and the electroencephalogram application areidentified, a voice instruction that prompts the user with the eyesclosed to image the registered pass-thought is output from the earphone20A (S24). In the earphone 20A, the measurement mode is changed from thetrigger action measurement mode to the electroencephalogram measurementmode.

When the user imagines the pass-thought according to the voiceinstruction, the brain wave signal is measured by the electrode of theearphone 20A (S25). In this example, since the user has imaged a cat,the brain wave signal including the waveform W22 in FIG. 6 is measured,and the pass-thought recognition process is started.

In this pass-thought recognition process, it is checked whether there isa signal having a similarity within a predetermined threshold value inthe brain wave signal recorded for each correspondingelectroencephalogram application.

In this example, since the waveform W22 of the brain wave signalmeasured at the time of authentication has a similarity within the rangeof the predetermined threshold value with the waveform W12 of the brainwave signal recorded as the brain wave signal of theelectroencephalogram personal authentication application, theelectroencephalogram personal authentication application is notifiedthat the brain wave signals match (S26).

Note that, in a case where there is no brain wave signal having asimilarity within a range of a predetermined threshold value with thewaveform W22 of the measured brain wave signal, the electroencephalogrampersonal authentication application is notified that the brain wavesignals do not match.

When the pass-thought recognition process is completed, a voicenotifying the user that the authentication is completed and that theeyes may be opened is output from the earphone 20A (S27). Further, themobile terminal 10A displays a screen including a message indicatingthat the authentication is completed by an image or a text (S27).

In this manner, the user performs an action or the like serving as atrigger action according to the user interface (UI) such as a voiceinstruction, or images a thing or the like used as a pass-thought,whereby the electroencephalogram personal authentication is performed.

Example of External Induction

In the above description, an example in which the user actively imagesthe pass-thought at the time of registration and authentication in theelectroencephalogram personal authentication is described, but thepass-thought imaged by the user may be induced from the outside. First,an example of electroencephalogram personal authentication using anauditory induction response will be described with reference to FIGS. 7and 8 .

FIG. 7 illustrates a registration example of electroencephalogrampersonal authentication using an auditory induction response.

The user operates the menu screen of the electroencephalogram personalauthentication application being activated on the mobile terminal 10A toselect registration of a trigger action and a pass-thought (S31). Notethat it is possible to select to use the auditory induction response onthis menu screen.

In response to this selection operation, the mobile terminal 10Apresents a message prompting the user to close the eyes and listen tothe sound output from the earphone 20A with an image, a text, a sound,or the like (S32).

When the user closes the eyes according to the message presented to themobile terminal 10A, the myoelectric signal corresponding to the actionis measured by the electrode of the earphone 20A and recorded as atrigger action signal (S33).

Thereafter, the induction sound is output from the earphone 20A to theuser with the eyes closed, and when the user responds to the inductionsound, the brain wave signal corresponding to the response is measuredby the electrode of the earphone 20A, and recorded in association withthe electroencephalogram personal authentication application and thetrigger action (S34).

When the pass-thought corresponding to the auditory induction responseis registered, a voice notifying the user that the registration iscompleted and that the eyes may be opened is output from the earphone20A (S35).

According to the voice instruction, when the user opens the eye, themyoelectric signal corresponding to the action is measured by theelectrode of earphone 20A, and is recorded as the trigger action signal(S36). That is, in this example, by recording a series of actions whenthe user closes the eyes and when the user opens the eyes, theopening/closing action of the eyes is recorded as the trigger action.

Note that it is possible to register the pass-thought according to theauditory induction response with higher accuracy by repeating theprocesses of steps S32 to S34 described above a plurality of times andchecking whether a waveform matches (or is similar to) the previouswaveform.

FIG. 8 illustrates an authentication example of electroencephalogrampersonal authentication using an auditory induction response.

The earphone 20A worn on the ear by the user is operating in a triggeraction measurement mode (S41). When the user opens and closes the eyes,a myoelectric signal corresponding to the action is detected as atrigger action signal, and a trigger action recognition process isstarted.

In this example, in the trigger action recognition process, since themyoelectric signal detected as the trigger action signal at the time ofauthentication has a similarity within the range of the predeterminedthreshold value with the myoelectric signal recorded as the triggeraction signal at the time of registration described above, the triggeraction is opening and closing of the eyes, and the correspondingelectroencephalogram application is the electroencephalogram personalauthentication application (S42).

Once the trigger action and electroencephalogram application areidentified, the measurement mode is changed to the electroencephalogrammeasurement mode. Then, the induction sound is output from the earphone20A to the user with the eyes closed, and when the user responds to theinduction sound, the brain wave signal corresponding to the response ismeasured by the electrode of the earphone 20A, and the ERP recognitionprocess is started (S43). The event-related potential (ERP) is a brainresponse (typological electrophysiological response to internal andexternal stimuli) measured in some form as a result of a thought orcognition of a user.

In this example, in the ERP recognition process, since the brain wavesignal measured at the time of authentication has a similarity within arange of a predetermined threshold value with the brain wave signalrecorded as the brain wave signal of the electroencephalogram personalauthentication application, the electroencephalogram personalauthentication application is notified that the brain wave signalsmatch.

When the ERP recognition process is completed, a sound (authenticationclear sound) indicating that the authentication is completed and a voicemaking notification that the eyes may be opened are output from theearphone 20A to the user (S44).

In this way, by using the auditory induction response, the user canrealize the electroencephalogram personal authentication only bylistening to the induction sound output from the outside without imagingthe pass-thought used for the authentication key by himself/herself.

Note that various sounds can be used as the induction sound, but an oddball task in which an abnormal sound is mixed with a normal sound may beused. The odd ball task is to present two or more kinds of sounds in arandom order with different appearance frequencies.

Next, an example of electroencephalogram personal authentication using avisual induction response will be described with reference to FIGS. 9and 10 .

FIG. 9 illustrates a registration example of electroencephalogrampersonal authentication using a visual induction response.

As in steps S31 to S33 of FIG. 7 , in steps S51 to S53 of FIG. 9 , whenthe user closes the eyes according to the message presented by themobile terminal 10A or the earphone 20A, the myoelectric signalcorresponding to the action is measured by the electrode of the earphone20A and recorded as the trigger action signal.

The user with the eyes closed is irradiated with the inductive lightusing the visual induction illumination function of the mobile terminal10A, and when the user responds to the inductive light, the brain wavesignal corresponding to the response is measured by the electrode of theearphone 20A, and recorded in association with the electroencephalogrampersonal authentication application and the trigger action (S54).

For example, as the visual induction illumination, flash illuminationwith a specific pulse is used, and flash is only required to be enabledat a speed at which the user is not conscious, but at which an inductionresponse occurs in the brain wave signal. Note that the visual inductionillumination function is described as a function of the mobile terminal10A, but may be provided as a function of another device.

As in steps S35 and S36 of FIG. 7 , in steps S55 to S56 of FIG. 9 ,after notification of the completion of the registration is provided,the myoelectric signal corresponding to the action when the user opensthe eyes is measured and recorded as the trigger action signal. That is,in this example, by recording a series of actions when the user closesthe eyes and when the user opens the eyes, the opening/closing action ofthe eyes is recorded as the trigger action.

Note that it is possible to register the pass-thought according to thevisual induction response with higher accuracy by repeating theprocesses of steps S52 to S54 described above a plurality of times andchecking whether a waveform matches (or is similar to) the previouswaveform.

FIG. 10 illustrates an authentication example of electroencephalogrampersonal authentication using a visual induction response.

As in steps S41 to S42 of FIG. 8 , in steps S61 to S62 of FIG. 10 , in acase where the myoelectric signal corresponding to the action when theuser opens and closes the eyes is detected as the trigger action signal,and has a similarity within a predetermined threshold value range withthe trigger action signal recorded at the time of registration, the ERPrecognition process is started (S63).

In this ERP recognition process, when the user responds to the inductivelight radiated by the mobile terminal 10A, the brain wave signalcorresponding to the response is measured. Therefore, in a case wherethe measured brain wave signal has a similarity within a predeterminedthreshold value range with the brain wave signal recorded as the brainwave signal of the electroencephalogram personal authenticationapplication, the electroencephalogram personal authenticationapplication is notified that the brain wave signals match.

As in step S44 of FIG. 8 , in step S64 of FIG. 10 , when theauthentication of the pass-thought is completed, a sound (authenticationclear sound) indicating that the authentication is completed and a voicemaking notification that the eyes may be opened are output from theearphone 20A to the user.

In this way, by using the visual induction response, the user canrealize the electroencephalogram personal authentication only byreceiving the radiation of the inductive light output from the outsidewithout imaging the pass-thought used for the authentication key byhimself/herself.

Note that continuous and sustained electroencephalogram personalauthentication may be performed when a pulse at a speed at which theuser is not conscious is inserted into a signal, for example, in a roomlight, and the signal matches the brain wave signal of the visualinduction response registered in advance. In addition, at this time, theconcentration of the user by the brain wave signal of the visualinduction response may be measured.

Furthermore, since the room light or the like is used, theelectroencephalogram personal authentication can be simultaneouslyperformed not only for one user but also for a plurality of users in theroom. For example, by providing the function of the electroencephalogrampersonal authentication as a function of the conference roomapplication, in a case where the conference room is used by a pluralityof users, the electroencephalogram personal authentication is performedon the users in the conference room, and it is possible to grasp theparticipation status of the conference.

In addition, in the above description, the electroencephalogram personalauthentication using the auditory induction response and the visualinduction response has been exemplified, but in addition to this, atactile induction response by vibration of a device such as the mobileterminal 10A, an olfactory induction response by a scent generationdevice, or the like may be used. Such a brain wave signal induced byhearing, vision, touch, or smell can also be said to be an event-relatedpotential (ERP).

Second Example

In the above description, the example in which the electroencephalogrampersonal authentication application is executed as theelectroencephalogram application is described, but the presenttechnology can be applied even in a case where anotherelectroencephalogram application is used. An example of the recallsearch (brain log search) using the electroencephalogram searchapplication will be described with reference to FIGS. 11 and 12 .

FIG. 11 illustrates a first example of the recall search using theelectroencephalogram search application.

In FIG. 11 , the user holds the mobile terminal 10A such as a smartphonein the hand, and wears on the ear the earphone 20A connected to themobile terminal 10A in a wireless or wired manner. In addition, in themobile terminal 10A, applications such as a social networking service(SNS) application and a news application are activated.

At this time, in a case where the user operates the mobile terminal 10Aand browses an article of an SNS application and the like, a scene inwhich there is an article of interest is assumed (S71).

In this scene, the brain wave signal of the user when browsing thearticle of interest is measured and recorded by the electrode of theearphone 20A.

Regarding the timing of recording the brain wave signal, for example,when an article fixedly displayed for a certain period of time or moreis detected on the mobile terminal 10A, it is estimated that the userhas been paying attention to and browsing the article, and the brainwave signal at that time is automatically recorded.

In this example, a brain wave signal including a waveform W71 in FIG. 11is measured and recorded as a brain wave signal of the user when anarticle fixedly displayed for a certain period of time or more isdetected. Recording of the brain wave signal can be associated with theelectroencephalogram search application. In addition, the informationregarding the recorded brain wave signal is associated with theinformation regarding the article of interest by information such ashistory information.

Thereafter, a scene is assumed in which the user tries to browse thearticle of interest again, but cannot remember which article it isalthough trying to search for the article of interest (S72). At thistime, it is assumed that the user cannot even think of a search word onwhich application the user has browsed.

At this time, when the user closes the eyes, the myoelectric signalcorresponding to the action is detected as a trigger action signal bythe electrode of the earphone 20A, and the trigger action recognitionprocess is started (S73). Note that, in this example, it is assumed thata user's eye closing action (eye closing) is registered in advance as atrigger action of the electroencephalogram search application.

In this trigger action recognition process, in a case where the detectedtrigger action signal has a similarity within a predetermined thresholdvalue range with the recorded trigger action signal, the trigger actionis eye closing, and the corresponding electroencephalogram applicationis an electroencephalogram search application.

When a trigger action and an electroencephalogram application areidentified, and a user with the eyes closed thinks of an image of anarticle of interest, a brain wave signal corresponding to the image ofthe article of interest is measured by an electrode of the earphone 20A,and a brain log search process is started (S73).

In this brain log search process, matching is performed as to whetherthere is a waveform having a similarity within a predetermined thresholdvalue range with the measured waveform of the brain wave signal amongthe waveforms of the brain wave signals recorded as the brain wavesignals of the electroencephalogram search application. In this example,since the waveform of the measured brain wave signal has a similaritywithin a range of a predetermined threshold value with the waveform W71of the recorded brain wave signal, the electroencephalogram searchapplication is notified of information regarding the brain wave signal.

In the mobile terminal 10A, the electroencephalogram search applicationidentifies information regarding the article of interest associated withthe information regarding the brain wave signal notification of which isto be provided on the basis of the information such as the historyinformation, and presents the article of interest on the basis of theinformation regarding the article of interest (S74). For example, thearticle of interest can be presented using a history option function orthe like of the mobile terminal 10A. In addition, at the time ofpresenting the article of interest, in a case where there is a pluralityof histories (records) in which the similarity of the waveform of thebrain wave signal falls within the range of the threshold value, aplurality of matching results may be presented in descending order ofthe similarity.

In this manner, it is possible to realize the recall search (brain logsearch) by matching the brain wave signal automatically recorded whenthe user browses the article with the brain wave signal when the userrecalled the storage of the article. As a result, even in a case wherethe user does not clearly remember the information regarding the articlethat the user has browsed, the user can browse the article again byrecalling the memory of the article in his/her head.

FIG. 12 illustrates a second example of the recall search using theelectroencephalogram search application.

In FIG. 12 , a scene is assumed in which the user is heading to adestination such as a meeting destination while confirming the mapapplication activated on the mobile terminal 10A held in the handoutdoors (S81).

In this scene, the brain wave signal of the user when the appearance ofthe building is confirmed on the way to the destination is measured andrecorded by the electrode of the earphone 20A.

Regarding the timing at which the brain wave signal is recorded, forexample, it is estimated that when a time for the user to stop for acertain period of time is detected after a map is displayed by a mapapplication activated on the mobile terminal 10A, the appearance of thebuilding is checked at that place, and the brain wave signal at thattime is automatically recorded.

In this example, a brain wave signal including a waveform W81 in FIG. 12is measured and recorded as a brain wave signal of the user when a timefor the user to stop for a certain period of time is detected. Recordingof the brain wave signal can be associated with the electroencephalogramsearch application. In addition, the information regarding the recordedbrain wave signal is associated with information (position informationor the like) regarding a place where the appearance of the building isestimated to have been confirmed on the basis of information such ashistory information.

Thereafter, a scene is assumed in which the user remembers theappearance of the building but cannot remember where the scene is seen(S82).

At this time, when the user closes the eyes, the myoelectric signalcorresponding to the action is measured by the electrode of the earphone20A, and the trigger action recognition process is started (S83). Notethat, also in this example, it is assumed that eye closing is registeredin advance as the trigger action.

In this trigger action recognition process, in a case where the detectedtrigger action signal has a similarity within a predetermined thresholdvalue range with the recorded trigger action signal, the trigger actionis eye closing, and the corresponding electroencephalogram applicationis an electroencephalogram search application.

When a trigger action and an electroencephalogram application areidentified, and a user with the eyes closed thinks of an image of animpressive building, a brain wave signal corresponding to the image ofthe impressive building is measured by the electrode of the earphone20A, and a brain log search process is started (S83).

In this brain log search process, matching is performed as to whetherthere is a waveform having a similarity within a predetermined thresholdvalue range with the measured waveform of the brain wave signal amongthe waveforms of the brain wave signals recorded as the brain wavesignals of the electroencephalogram search application. In this example,since the waveform of the measured brain wave signal has a similaritywithin a range of a predetermined threshold value with the waveform W81of the recorded brain wave signal, the electroencephalogram searchapplication is notified of information regarding the brain wave signal.

In the mobile terminal 10A, the electroencephalogram search applicationidentifies a place (position information or the like) associated withthe information regarding the brain wave signal notification of which isto be provided on the basis of the information such as the historyinformation, and presents a map of the place, information regarding abuilding, or the like (S84). For example, a map of a specific place orinformation regarding a building can be presented by using a historyoption function of the mobile terminal 10A.

In this manner, it is possible to realize the recall search (brain logsearch) by matching the brain wave signal automatically recorded whenthe user visits a specific place with the brain wave signal when theuser recalls the memory of a building or the like with an impression. Asa result, even in a case where the user does not clearly memorizeinformation regarding a building or the like with an impression at theplace where the user has visited, the user can browse the informationregarding the building or the like by using the mobile terminal 10A byrecalling the memory of the building or the like in his/her head.

Note that FIGS. 11 and 12 illustrate a case where the trigger action ofthe electroencephalogram search application is eye closing forconvenience of description. However, for example, another action such asblinking twice may be registered. That is, trigger actions of theelectroencephalogram personal authentication application and theelectroencephalogram search application can be different. In addition,in the above-described example, the brain wave signal regarding athought of the user for the same object such as a specific article or aspecific place is described as the brain wave signal to be subjected tothe matching process. The same object can include information, such as astill image, a moving image, a contact address, music, or anapplication, that can be presented by a device such as the mobileterminal 10A and that is provided by a specific medium.

(System Configuration)

FIG. 13 illustrates a configuration example of an electroencephalogramsystem to which the present technology is applied.

The electroencephalogram system is a system capable of providing variousservices using a brain wave signal measured from the head of the user.In FIG. 13 , the electroencephalogram system includes a terminal device10 and a measurement device 20.

The terminal device 10 is an electronic apparatus such as the mobileterminal 10A such as the above-described smartphone, a game machine, acontroller, a personal computer (PC), a display device, or a portablemusic player. In FIG. 13 , the terminal device 10 includes a controlunit 101 and a communication unit 102.

The control unit 101 is a main control device that controls variousoperations and performs various types of arithmetic processing. Thecontrol unit 101 includes a processor such as a central processing unit(CPU). The control unit 101 can activate and execute theelectroencephalogram application.

The communication unit 102 communicates with other devices such as themeasurement device 20 under the control of the control unit 101. Thecommunication unit 102 is configured as a communication modulecompatible with wireless communication or wired communication conformingto a predetermined communication scheme.

For example, the wireless communication includes wireless communicationaccording to a short-distance wireless communication standard such asBluetooth (registered trademark) or near field communication (NFC),wireless communication such as a wireless local area network (LAN), andcellular communication such as LTE-Advanced or a fifth generation (5G).In addition, the wired communication includes communication compatiblewith a communication interface such as a high definition multimediainterface (HDMI) (registered trademark).

The measurement device 20 is an electronic apparatus such as theearphone 20A described above, a head mounted display (HMD) 20B describedbelow, and a wearable terminal such as a glasses-type wearable device.

In FIG. 13 , the measurement device 20 includes electrodes 201-1 to201-n (n: an integer of one or more), a reference electrode 202, asensor unit 203, a sound output unit 204, a display unit 205, an inputsignal processing unit 206, a signal recording unit 207, a communicationunit 208, an output processing unit 209, and a battery 210.

The electrodes 201-1 to 201-n are measurement electrodes that measurebiometric signals. The reference electrode 202 is a reference electrodethat measures a reference potential used for calculating a differencefrom the potentials measured by the electrodes 201-1 to 201-n.

The electrodes 201-1 to 201-n and the reference electrode 202 areattached so as to be in close contact with parts such as the head andthe ear of the user. The biometric signals measured by the electrodes201-1 to 201-n and the reference electrode 202 are supplied to the inputsignal processing unit 206.

The electrode may include a ground electrode. The ground electrode hereis not a common ground electrode (electrode having a ground potential),but refers to an electrode having a potential serving as a referencepotential for a user. In the following description, the electrodes 201-1to 201-n will be simply referred to as the electrode 201 in a case whereit is not particularly necessary to distinguish them.

The sensor unit 203 performs sensing of space information, timeinformation, and the like, and supplies a sensor signal obtained as aresult of the sensing to the input signal processing unit 206. Forexample, the sensor unit 203 includes an acceleration sensor 221 and agyro sensor 222.

The acceleration sensor 221 measures accelerations in three directionsof the XYZ axes. The gyro sensor 222 measures angular velocities aroundthree axes of the XYZ axes. Note that an inertial measurement unit (IMU)may be provided to measure a three-dimensional acceleration and anangular velocity with a three directions accelerometer and a three axesgyroscope.

The sound output unit 204 outputs a sound corresponding to the soundsignal from the output processing unit 209. The sound output unit 204includes a mechanism such as a driver unit constituting an earphone, aspeaker, or the like.

The display unit 205 displays an image corresponding to the image signalfrom the output processing unit 209. The display unit 205 includes apanel unit such as a liquid crystal panel or an organic light emittingdiode (OLED) panel, a signal processing unit, and the like.

The input signal processing unit 206 processes the biometric signalsfrom the electrodes 201-1 to 201-n and the reference electrode 202, andreads a myoelectric signal which is a weak signal generated in a nervewhen a human moves a muscle or a brain wave signal which is a signalcorresponding to electrical activity generated from a human brain.

The input signal processing unit 206 performs a predetermined signalprocess on the basis of the read myoelectric signal or the read brainwave signal. Note that the input signal processing unit 206 may use thesensor signal from the sensor unit 203 together with the myoelectricsignal at the time of detecting the trigger action.

That is, the input signal processing unit 206 has a function as adetection unit that detects a brain wave included in the biometricsignal of the user and detects a motion based on information (such as amyoelectric signal and a sensor signal) other than the brain waveincluded in the biometric signal. In addition, the input signalprocessing unit 206 has a function as a processing unit that performsthe predetermined process based on brain wave.

The input signal processing unit 206 records data related to the readmyoelectric signal or the read brain wave signal in the signal recordingunit 207. The input signal processing unit 206 performs matching betweenthe read myoelectric signal or the read brain wave signal and themyoelectric signal or brain wave signal recorded in the signal recordingunit 207, and supplies data related to the matching result to thecommunication unit 208.

The signal recording unit 207 records data related to various signalsunder the control of the input signal processing unit 206. The signalrecording unit 207 is configured as an auxiliary storage device such asa semiconductor memory. The signal recording unit 207 may be configuredas an internal storage or may be an external storage such as a memorycard.

The communication unit 208 communicates with another device such as theterminal device 10. The communication unit 208 transmits the data fromthe input signal processing unit 206 to the terminal device 10. Inaddition, the communication unit 208 receives data transmitted from theterminal device 10 and supplies the data to the output processing unit209.

The communication unit 208 is configured as a communication modulecompatible with wireless communication or wired communication conformingto a predetermined communication scheme. For example, the wirelesscommunication includes wireless communication according to ashort-distance wireless communication standard such as Bluetooth(registered trademark), wireless communication such as a wireless LAN,and cellular communication such as LTE-Advanced or 5G. The wiredcommunication includes communication compatible with a communicationinterface such as an HDMI (registered trademark).

The output processing unit 209 processes the data from the communicationunit 208, supplies a sound signal to the sound output unit 204, andsupplies an image signal to the display unit 205.

The battery 210 is detachably attached to the measurement device 20, andsupplies power to each unit of the measurement device 20 via apredetermined terminal.

Note that the configuration illustrated in FIG. 13 is an example of theterminal device 10 and the measurement device 20, and the illustratedcomponents may be removed or new components may be added.

For example, in the measurement device 20, there is a case where thesensor unit 203 including the acceleration sensor 221, the gyro sensor222, and the like is not mounted. In addition, in a case where themeasurement device 20 is the earphone 20A, the display unit 205 is notmounted. Furthermore, in a case where the measurement device 20 is anHMD 20B, the sound output unit 204 may not be mounted.

In addition, although the configuration of the terminal device 10 is theminimum configuration, an output processing unit, a sound output unit, adisplay unit, a signal recording unit, a battery, and the like may beprovided as in the measurement device 20.

Arrangement Example of Electrodes

FIG. 14 illustrates an arrangement example of the electrode 201 and thereference electrode 202 provided in the earphone 20A.

As illustrated in FIG. 14 , in the earphone 20A, four electrodes of theelectrodes 201-1 to 201-3 and the reference electrode 202 are disposedat substantially equal intervals on the same circumference around theportion that outputs a sound on the face in contact with the ear portionof the user, and the biometric signal of the user can be measured.

FIG. 15 illustrates an arrangement example of the electrode 201 and thereference electrode 202 provided in the HMD 20B.

As illustrated in FIG. 15 , in the HMD 20B, eight electrodes 201-1 to201-8 are disposed at substantially equal intervals in a straight lineon the face in contact with the forehead of the user, and the referenceelectrode 202 is disposed, away from the eight electrodes, at apredetermined position on the face in contact with the back of the head,and the biometric signal of the user can be measured.

Note that, in the perspective view of FIG. 15 , among the electrodes201-1 to 201-8, three sensors of the electrodes 201-6 to 201-8 are notillustrated because they are in blind spots. In addition, forconvenience of explanation, the electrode 201 is illustrated to appearin the appearance of the HMD 20B, but is actually provided on the facein contact with the forehead of the user, that is, at a position notvisible in the perspective view of FIG. 15 .

Configuration Example of Table

FIG. 16 illustrates a configuration example of a table recorded in thesignal recording unit 207 of FIG. 13 .

The signal recording unit 207 records a trigger action signal recordingtable 231, a trigger action-specific electroencephalogram applicationtable 232, and a brain wave signal recording table 233.

The trigger action signal recording table 231 is a table in which dataof trigger action signals including a combination of head myoelectricpotentials by parts such as the eyes, the jaw, and the neck of the useris recorded.

As illustrated in FIG. 17 , in the trigger action signal recording table231, the trigger action and the trigger action signal are recorded inassociation with each other.

For example, data of the myoelectric signal #1 is recorded as a triggeraction signal of a trigger action (Trigger Action #1) that is eyeclosing, data of the myoelectric signal #2 is recorded as a triggeraction signal of a trigger action (Trigger Action #2) that is blinkingtwice, and data of the myoelectric signal #3, the acceleration signal#1, and the angular velocity signal #1 are recorded as a trigger actionsignal of a trigger action (Trigger Action #3) that is eye closing aftertwo blinks of both eyes with the neck down.

Note that a representative trigger action is eye closing, and the brainwave signal is read in a state where the user closes the eyes, but thebrain wave signal may be read in a state where the eyes are opened, thatis, the user opens the eyes. In the example of FIG. 17 , only the dataof the myoelectric signal is recorded as the trigger action signal forthe trigger action of Trigger Action #1 and Trigger Action #2, but dataindicating that the input values of the acceleration signal and theangular velocity signal are 0 (0 input value) may be recorded.

The trigger action-specific electroencephalogram application table 232is a table in which the data of the electroencephalogram application foreach trigger action recorded in the trigger action signal recordingtable 231 is recorded.

For example, as illustrated in FIG. 18 , in the trigger action-specificelectroencephalogram application table 232, Trigger Action #1 in FIG. 17is associated with the electroencephalogram personal authenticationapplication, and Trigger Action #2 in FIG. 17 is associated with theelectroencephalogram search application.

The authentication in the electroencephalogram personal authenticationapplication is not limited to the user's identity authentication. Inaddition, this electroencephalogram personal authentication may beapplied to personal authentication implemented in Internet banking,Internet shopping, user selection at the time of viewing a contentdistributed via the Internet, an answer of a questionnaire via theInternet, a test or an interview using the Internet, or the like.Further, the present invention may be applied to personal authenticationof a professional driver of a vehicle such as an automobile, anairplane, or a train.

The brain wave signal recording table 233 is a table in which data of abrain wave signal registered for each electroencephalogram applicationrecorded in the trigger action-specific electroencephalogram applicationtable 232 is recorded.

As illustrated in FIG. 19 , in the brain wave signal recording table233, an electroencephalogram application and a brain wave signal arerecorded in association with each other.

For example, data such as the brain wave signal #11, the brain wavesignal #12, and the brain wave signal #13 is recorded as the brain wavesignal of the electroencephalogram personal authentication application,and data such as the brain wave signal #21, the brain wave signal #22,and the brain wave signal #23 is recorded as the brain wave signal ofthe electroencephalogram search application.

Another Configuration Example

FIGS. 20 to 22 illustrate other configuration examples of theelectroencephalogram system to which the present technology is applied.

FIG. 20 illustrates a first example of another configuration of theelectroencephalogram system to which the present technology is applied.

In FIG. 20 , the electroencephalogram system includes the terminaldevice 10 and the measurement device 20. The same portions as those inthe configuration example in FIG. 13 are denoted by the same referencenumerals, and the description thereof will be omitted because it isrepeated.

In FIG. 20 , the terminal device 10 includes an input signal processingunit 103, a signal recording unit 104, an output processing unit 105, asound output unit 106, and a display unit 107 in addition to the controlunit 101 and the communication unit 102. The measurement device 20includes the electrodes 201-1 to 201-n, the reference electrode 202, thesensor unit 203, the sound output unit 204, the display unit 205, thecommunication unit 208, and the battery 210.

That is, the configuration example of FIG. 20 is different from theconfiguration example of FIG. 13 in that the input signal processingunit 103, the signal recording unit 104, and the output processing unit105 are provided in the terminal device 10 instead of the input signalprocessing unit 206, the signal recording unit 207, and the outputprocessing unit 209 in the measurement device 20 in FIG. 13 .

In the measurement device 20, the communication unit 208 transmits thebiometric signals from the electrodes 201-1 to 201-n and the referenceelectrode 202 to the terminal device 10.

On the other hand, in the terminal device 10, the communication unit 102receives the biometric signal transmitted from the measurement device 20and supplies the biometric signal to the input signal processing unit103 via the control unit 101.

The input signal processing unit 103 processes the biometric signalreceived by the communication unit 102, reads the myoelectric signal orthe brain wave signal, and performs a predetermined signal process. Theinput signal processing unit 103 records data related to the readmyoelectric signal or the read brain wave signal in the signal recordingunit 104 such as a semiconductor memory.

The input signal processing unit 103 performs matching between the readmyoelectric signal or the read brain wave signal and the myoelectricsignal or brain wave signal recorded in the signal recording unit 104,and supplies data related to the matching result to the communicationunit 102 or the output processing unit 105 via the control unit 101.

The output processing unit 105 processes the data from the input signalprocessing unit 103 and supplies the processed data to the communicationunit 102. The communication unit 102 transmits the data from the controlunit 101 to the measurement device 20. In the measurement device 20, thecommunication unit 208 receives data transmitted from the terminaldevice 10, supplies a sound signal to the sound output unit 204, andsupplies an image signal to the display unit 205.

The sound output unit 204 outputs a sound corresponding to the soundsignal from the communication unit 208. The display unit 205 displays animage corresponding to the image signal from the communication unit 208.

Note that the output processing unit 105 may process data input thereto,supply a sound signal to the sound output unit 106, and supply an imagesignal to the display unit 107. At this time, the sound output unit 106outputs a sound corresponding to the sound signal from the outputprocessing unit 105. In addition, the display unit 107 displays an imagecorresponding to the image signal from the output processing unit 105.

FIG. 21 illustrates a second example of another configuration of theelectroencephalogram system to which the present technology is applied.

In FIG. 21 , the electroencephalogram system includes the terminaldevice 10, the measurement device 20, and a server 30. In FIG. 21 , thesame components as those in the configuration example of FIG. 13 aredenoted by the same reference numerals, and the description thereof willbe omitted because it is redundant.

In FIG. 21 , the terminal device 10 includes the communication unit 102.The measurement device 20 includes the electrodes 201-1 to 201-n, thereference electrode 202, the sensor unit 203, the sound output unit 204,the display unit 205, the communication unit 208, and the battery 210.The server 30 includes a control unit 301, a communication unit 302, aninput signal processing unit 303, a signal recording unit 304, and anoutput processing unit 305.

That is, the configuration of FIG. 21 is different from theconfiguration example of FIG. 13 in that the input signal processingunit 303, the signal recording unit 304, and the output processing unit305 are provided in the server 30 instead of the input signal processingunit 206, the signal recording unit 207, and the output processing unit209 in the measurement device 20 in FIG. 13 .

In the measurement device 20, the communication unit 208 transmits thebiometric signals from the electrodes 201-1 to 201-n and the referenceelectrode 202 to the terminal device 10. In the terminal device 10, thecommunication unit 102 receives the biometric signal transmitted fromthe measurement device 20 to transmit the biometric signal to the server30 via the network 40.

In the server 30, the communication unit 302 receives the biometricsignal transmitted from the terminal device 10 and supplies thebiometric signal to the input signal processing unit 303 via the controlunit 301.

The input signal processing unit 303 processes the biometric signalreceived by the communication unit 302, reads the myoelectric signal orthe brain wave signal, and performs the predetermined process. The inputsignal processing unit 303 records data related to the read myoelectricsignal or the read brain wave signal in the signal recording unit 304such as a semiconductor memory or a hard disk drive (HDD).

The input signal processing unit 303 performs matching between the readmyoelectric signal or the read brain wave signal and the myoelectricsignal or brain wave signal recorded in the signal recording unit 304,and supplies data related to the matching result to the communicationunit 302 or the output processing unit 305 via the control unit 301.

The output processing unit 305 processes the data from the input signalprocessing unit 303 and supplies the processed data to the communicationunit 302. The communication unit 302 transmits the data from the controlunit 301 to the measurement device 20 via a network 40 and the terminaldevice 10. In the measurement device 20, the communication unit 208receives data transmitted from the server 30, supplies a sound signal tothe sound output unit 204, and supplies an image signal to the displayunit 205.

The sound output unit 204 outputs a sound corresponding to the soundsignal from the communication unit 208. The display unit 205 displays animage corresponding to the image signal from the communication unit 208.

The network 40 includes a communication network such as the Internet, anintranet, or a mobile phone network, and enables interconnection betweendevices using a communication protocol such as a transmission controlprotocol/internet protocol (TCP/IP).

FIG. 22 illustrates a third example of another configuration of theelectroencephalogram system to which the present technology is applied.

In FIG. 22 , the electroencephalogram system is configured by themeasurement device 20 and the server 30. The same portions as those inthe configuration examples of FIGS. 13 and 21 are denoted by the samereference numerals, and the description thereof will be omitted becauseit is repeated.

In FIG. 22 , the measurement device 20 includes a control unit 211 inaddition to the electrodes 201-1 to 201-n, the reference electrode 202,the sensor unit 203, the sound output unit 204, the display unit 205,the input signal processing unit 206, the signal recording unit 207, thecommunication unit 208, the output processing unit 209, and the battery210. The server 30 includes the control unit 301, the communication unit302, and the signal recording unit 304.

That is, the configuration of FIG. 22 is different from theconfiguration example of FIG. 13 in that (the control unit 101 of) theterminal device 10 is not provided, but the control unit 211 is providedin the measurement device 20, and the measurement device 20 also has thefunction of the terminal device 10.

In addition, the configuration of FIG. 22 is different from theconfiguration example of FIG. 21 in that only the signal recording unit304 in addition to the control unit 301 and the communication unit 302is provided in the server 30, and the server 30 operates as a databaseserver on a cloud.

Although other configuration examples of the electroencephalogram systemto which the present technology is applied have been described above,the configurations illustrated in FIGS. 20 to 22 are merely examples,and other configurations may be used. That is, in theelectroencephalogram system, the input signal processing unit(103,206,303), the signal recording unit (104,207,304), and the outputsignal processing unit (105,209,305) may be provided in any of theterminal device 10, the measurement device 20, and the server 30.

In addition, the terminal device 10, the measurement device 20, and theserver 30 can also be said to be control devices since functions of theinput signal processing unit (103,206,303) and the output signalprocessing unit (105,209,305) are each implemented by a processor suchas a CPU executing a program.

(Flow of Processing)

The registration process of the trigger action and the pass-thought willbe described with reference to the flowcharts of FIGS. 23 and 24 .

In this registration process, the terminal device 10 or the measurementdevice 20 confirms the electroencephalogram application for registeringthe trigger action with respect to the user (S111), and afterdesignating the electroencephalogram application to be registered (“Yes”in S112), the process proceeds to step S113.

In step S113, the input signal processing unit 206 starts reading thetrigger action signal detected as the myoelectric signal.

In step S114, the input signal processing unit 206 processes thebiometric signal from the electrode 201 and the reference electrode 202,and determines whether the trigger action signal has been read. In thedetermination process of step S114, the process proceeds to step S115after it is determined that the trigger action signal has been read.

In step S115, the input signal processing unit 206 records the readtrigger action signal in the signal recording unit 207.

In step S116, the output processing unit 209 controls the sound outputunit 204 or the display unit 205 to present a user interface (UI)prompting the trigger action again to the user, and the input signalprocessing unit 206 starts to read the trigger action signal.

In step S117, the input signal processing unit 206 processes thebiometric signal from the electrode 201 and the reference electrode 202,and determines whether the trigger action signal has been read. In thedetermination process of step S117, the process proceeds to step S118after it is determined that the trigger action signal has been read.

In step S118, the input signal processing unit 206 calculates asimilarity between the trigger action signals read in the process ofsteps S114 and S117.

In step S119, the input signal processing unit 206 determines whetherthe calculated similarity of the trigger action signals falls within apredetermined threshold value range.

In a case where it is determined in the determination process of stepS119 that the similarity falls within the predetermined threshold valuerange, the process proceeds to step S120. In step S120, the input signalprocessing unit 206 records the trigger action signal and the designatedelectroencephalogram application in the table of the signal recordingunit 207.

For example, in the trigger action signal recording table 231 of FIG. 17, a myoelectric signal is recorded as a trigger action signal inassociation with a trigger action such as eye closing or blinking twice.In addition, in the trigger action-specific electroencephalogramapplication table 232 of FIG. 18 , a designated electroencephalogramapplication such as an electroencephalogram personal authenticationapplication is recorded in association with a trigger action such as eyeclosing or blinking twice.

On the other hand, in a case where it is determined in the determinationprocess of step S119 that the similarity is out of the range of thepredetermined threshold value, the process proceeds to step S121. Instep S121, the output processing unit 209 controls the sound output unit204 or the display unit 205, and presents a user interface (UI)prompting the trigger action again to the user. Then, the processreturns to step S113, and the subsequent processes are repeated.

When the process of step S120 ends, the process proceeds to step S122 ofFIG. 24 . In step S122, the output processing unit 209 controls thesound output unit 204 or the display unit 205 to present a userinterface (UI) prompting an image of a pass-thought.

In step S123, the input signal processing unit 206 starts reading thebrain wave signal.

In step S124, the input signal processing unit 206 processes thebiometric signal from the electrode 201 and the reference electrode 202,and determines whether the brain wave signal has been read. In thedetermination process of step S124, the process proceeds to step S125after it is determined that the brain wave signal has been read.

In step S125, the input signal processing unit 206 records the readbrain wave signal in the signal recording unit 207.

In step S126, the output processing unit 209 controls the sound outputunit 204 or the display unit 205 to present the user with a userinterface (UI) that prompts an image of the pass-thought again, and theinput signal processing unit 206 starts to read the brain wave signal.

In step S127, the input signal processing unit 206 processes thebiometric signal from the electrode 201 and the reference electrode 202,and determines whether the brain wave signal has been read. In thedetermination process of step S127, the process proceeds to step S128after it is determined that the brain wave signal has been read.

In step S128, the input signal processing unit 206 calculates asimilarity of the brain wave signals read by the process in steps S124and S127.

In step S129, the input signal processing unit 206 determines whetherthe calculated similarity of the brain wave signals falls within apredetermined threshold value range.

In a case where it is determined in the determination process of stepS129 that the similarity falls within the predetermined threshold valuerange, the process proceeds to step S130. In step S130, the input signalprocessing unit 206 records the brain wave signal and the designatedelectroencephalogram application in the brain wave signal recordingtable 233.

For example, in the brain wave signal recording table 233 of FIG. 19 , abrain wave signal is recorded in association with a designatedelectroencephalogram application such as an electroencephalogrampersonal authentication application.

On the other hand, in a case where it is determined in the determinationprocess of step S129 that the similarity is out of the range of thepredetermined threshold value, the process proceeds to step S131. Instep S131, the output processing unit 209 controls the sound output unit204 or the display unit 205 to present the user with a user interface(UI) that prompts the image of a pass-thought again. Then, the processreturns to step S123, and the subsequent processes are repeated.

When the process of step S130 ends, the registration process of thetrigger action and the pass-thought ends.

Next, the personal electroencephalogram authentication process will bedescribed with reference to the flowchart of FIG. 25 .

In step S151, the input signal processing unit 206 determines whether atrigger action signal has been detected as a myoelectric signal on thebasis of the biometric signals from the electrode 201 and the referenceelectrode 202. In the determination process of step S151, the processproceeds to step S152 after detecting the trigger action signal.

In step S152, the input signal processing unit 206 performs matchingbetween the input trigger action signal and the registration triggeraction signal with reference to the trigger action signal recordingtable 231 of the signal recording unit 207.

In step S153, the input signal processing unit 206 determines whetherthere is a matched trigger action signal on the basis of the matchingresult of the trigger action signal.

In a case where it is determined in the determination process of stepS153 that there is no matched trigger action signal, the processproceeds to step S154. In step S154, notification of an error isprovided to the electroencephalogram application executed by theterminal device 10 or the measurement device 20.

Note that, in a case where retry is defined by the electroencephalogramapplication, the process returns to step S151, the subsequent processesare repeated, and matching of the trigger action signal is retried.

On the other hand, in a case where it is determined in the determinationprocess of step S153 that there is a matched trigger action signal, theprocess proceeds to step S155. In step S155, the input signal processingunit 206 reads the brain wave signal by the electroencephalogramapplication corresponding to the trigger action.

In step S156, the input signal processing unit 206 performs matchingbetween the input brain wave signal and the registered brain wave signalrecorded in the brain wave signal recording table 233 by an algorithmaccording to the corresponding electroencephalogram application.

In step S157, the input signal processing unit 206 determines whetherthere is a matched brain wave signal on the basis of the matching resultof the brain wave signal.

In a case where it is determined in the determination process of stepS157 that there is no matched brain wave signal, the process proceeds tostep S158. In step S158, notification of an error is provided to theelectroencephalogram application executed by the terminal device 10 orthe measurement device 20.

Note that, in a case where retry is defined by the electroencephalogramapplication, the process returns to step S155, the subsequent processesare repeated, and the matching of the brain wave signal is retried.

On the other hand, in a case where it is determined in the process ofstep S157 that there is a matched brain wave signal, the processproceeds to step S159. In step S159, notification of the matching resultis provided to the electroencephalogram application executed by theterminal device 10 or the measurement device 20. As a result, in theterminal device 10 or the measurement device 20, theelectroencephalogram application executes various types of processingaccording to the matching result.

In step S160, it is determined whether to end the action. In a casewhere it is determined in the determination process of step S160 thatthe action is not to be ended, the process returns to step S151, and thesubsequent processes are repeated. In addition, in a case where it isdetermined in the determination process of step S160 that the action isto be ended, the personal electroencephalogram authentication process isended.

As the trigger action described above, a response of an event-relatedpotential (ERP) may be used. In a case where the event-related potentialis used as the trigger action, for example, the following sequence (S91to S94) is obtained.

(1) An event inducing the event-related potential is presented (S91).

(2) Generation of the event-related potential in the user's brain waveis identified as a trigger action (S92).

(3) An instruction or the like for the pass-thought is presented (S93).

(4) Processing related to the pass-thought or the like is performed onthe basis of a waveform of a brain wave of the user for presentation ofan instruction or the like (S94).

In step S91, as the presentation of the event, for example, generationof a specific sound, presentation of a human face, presentation thatinduces a sense of discomfort, or the like is performed. Thesepresentations correspond to types of event-related potentials. Theevent-related potential includes, for example, N170 in which a brainwave response occurs after about 170 milliseconds for recognition of aface, MMN regarding acoustic deviation for auditory stimulation, P300 inwhich a brain wave response occurs after about 300 milliseconds fromrecognition of an event causing unnatural or uncomfortable feeling, andN400 in which a brain wave response occurs after about 400 millisecondsfrom recognition of deviation in meaning in language.

Since the event-related potentials such as N170, MMN, P300, and N400 aregenerated by the presentation of the event, the generation of theevent-related potential is recognized as the trigger action in step S92.

That is, in a case where the user is conscious of the presentationevent, the event-related potential serving as the trigger action isgenerated, but in a case where the user is not conscious of thepresentation event, the event-related potential does not occur, and theevent-related potential does not serve as the trigger action.

This can be understood as, for example, depending on whether or not theuser pays attention to the object content, and in a case where the userdoes not pay attention, such as looking at another content, a responseof the event-related potential does not occur. This is effective in acase where the user pays attention to the object content and wants toperform processing related to the subsequent pass-thought. Conversely,in a case where the user does not pay attention to the object content,such processing can be suppressed in a case where the user does not wantto perform the processing related to the subsequent pass-thought.

In addition, by using the response of the event-related potential, theprocessing can be performed by completing the brain wave of the usereven when there is not an explicit action of the user such as blinking.For example, the subsequent processing can be performed only byembedding the event that induces the event-related potential as thetrigger action so as not to be conspicuous (casually) in the objectcontent without an explicit instruction.

As in steps S24, S25 to S26 (FIG. 6 ) described above, in steps S93 andS94, processing such as a pass-thought recognition process is performedon the basis of the waveform of the user's brain wave for thepresentation of the instruction or the like.

Note that, in the above description, an example in which a user-specificresponse including an event-related potential is used is described.However, this is electroencephalogram acquisition for performing theprocess related to the pass-thought or the like, and description madehere is different from an example described above, and is not foracquiring the brain wave of the user for a process related to thepass-thought or the like, but an example in which an event-relatedpotential of the brain wave of the user is used as a trigger action.

As described above, the measurement device 20 includes the electrode201, the reference electrode 202, and the sensor unit 203 including theacceleration sensor 221 and the gyro sensor 222 in order to read thebrain wave signal and the myoelectric signal from the head of the user,and records the trigger action (any combination of movements of botheyelids, eyeballs, a jaw, and the like of the user and a movement of thehead) for each electroencephalogram application using the brain wave,and the user selectively uses the trigger action, so that reading of thebrain wave signal immediately after the trigger action is performed onlyfor the corresponding electroencephalogram application. Therefore, theconvenience and accuracy of the user interface by theelectroencephalogram input can be improved.

For example, in a case where the trigger action is “eye closing” and theelectroencephalogram application is “electroencephalogram personalauthentication application”, it is possible to naturally transition fromthe action of closing the eyes to the electroencephalogram measurementwith less artifact by using the head myoelectric potential generatedwhen the user closes the eyes as a trigger. Specifically, with thisflow, the user can naturally perform the resting state and the action asa series of flows.

That is, in the present technology, the action of the user is detectedon the basis of the myoelectric signal (information other than a brainwave) which is a biometric signal having a larger variation range of thepotential difference than the brain wave signal measured from the user,and in a case where the action of the user is a predetermined action,the predetermined process based on the brain wave signal measured fromthe user is performed. Therefore, a more convenient electroencephalograminput user interface can be provided. In addition, it is possible torealize processing related to brain waves such as electroencephalogrampersonal authentication having a natural user interface that is easy forall people to understand. Note that the artifact is so-called noise atthe time of measuring a biometric signal such as a brain wave signal ora myoelectric signal.

In general, the myoelectric signal of the head used for the triggeraction is larger than the brain wave signal and is characteristic.Therefore, in the measurement device 20, it is possible to contribute tolow power consumption by performing operation in the trigger actionmeasurement mode at the normal time and setting the time resolution andthe amplification factor of the signal amplifier at the time of readingthe myoelectric signal to be lower than those at the time of reading thebrain wave signal in the electroencephalogram measurement mode.

2. Second Embodiment

The biometric personal authentication is personal authenticationperformed using information regarding a physical feature or a behavioralfeature of an individual as a key for personal authentication. Normally,in this type of personal authentication, first, key information isregistered on the basis of biometric information about the user at thetime of registration, and then, at the time of authentication, thebiometric information about the user is acquired, matching it with thebiometric authentication that has been originally registered isperformed, and a process of permitting authentication is performed in acase where there is a certain level of correlation or greater.

A specific brain wave for a specific stimulus such as hearing or visionis referred to as an event-related potential, and various studies havebeen conducted. As described above, N170, MMN, P300, N400, and the likeare known as the event-related potentials.

In addition, authentication can be performed by detecting waveforms ofthese event-related potentials and, in addition, response componentspeculiar to the user. For example, in a case where a sound or a videofor giving a stimulus to the user is used as an induction medium, a userindividually outputs a characteristic and unique response brain wave fora specific induction medium, so that there is electroencephalogrampersonal authentication for authenticating an individual by performingmatching it with a past response brain wave.

Various techniques have been proposed as personal authenticationtechniques using event-related potentials. However, in the currenttechnology, since the personal authentication is performed on the basisof the registration result of the brain wave at the event-relatedpotential performed for the first time, there remains a risk of spoofingin a case where the information about the induction medium and theresponse brain wave leaks to the outside due to hacking or the like.

Therefore, in the present technology, the induction medium is generatedrandomly with unpredictable contents, and the above-describedauthentication and registration process is continuously performed in aform compatible with the service, so that the security is enhanced whilechanging the induction medium, and the risk of spoofing due to leakageof confidential information such as a key is reduced. Hereinafter, asecond embodiment will be described with reference to the drawings.

First Example

FIGS. 26 and 27 illustrate an authentication registration example ofelectroencephalogram personal authentication using auditory induction.

In FIGS. 26 and 27 , the user wears on the ear an earphone 21A connectedto a mobile terminal 11A such as a smartphone in a wireless or wiredmanner. The earphone 21A is provided with a plurality of electrodes, andcan measure a brain wave signal from the head of the user.

Although details will be described later, an fNIRS signal measured by ameasurement method applying the principle of optical functional imagingin which brain functions are non-invasively mapped from the scalp usingnear-infrared light, which is referred to as functional near-infraredspectroscopy (fNIRS), may be measured. That is, cerebral blood flowinformation measured using a measurement method for measuring a cerebralblood flow such as fNIRS may be used in addition to the brain wavesignal as brain wave information. Examples of a method for measuring acerebral blood flow include NIRS and MRI, fNIRS is included in NIRS, andfMRI is included in MRI. In the following description, a case wherefNIRS is used will be described as an example of a method of measuring acerebral blood flow.

FIG. 26 illustrates an authentication registration example of N-th (N:an integer of one or more) electroencephalogram personal authentication.

When an induction sound is output from the earphone 21A, and the userresponds to the auditory induction by the induction sound, a brain wavesignal corresponding to the response is measured (S211).

At this time, different sounds are output in the first half and thesecond half of the induction sound. That is, the induction sound is acombination of the registered sound and the new sound, and a differentcombination of sounds is replayed each time.

The first half induction sound TS11 is an induction sound in which abrain wave signal is registered last time ((N−1)-th time), and thecurrent (N-th) authentication is performed using a brain wave signalcorresponding to a response of the user to the induction sound. Thesecond half induction sound TS12 is an induction sound that is newly andrandomly generated, and a brain wave signal corresponding to a responseof the user to the induction sound is registered for use in the next((N+1)-th) authentication. Note that the induction sound TS12 is notlimited to a newly randomly generated induction sound, and may bereplayed by randomly selecting an induction sound generated in advance.

In this example, the brain wave signal for authentication including awaveform W111 corresponding to the induction sound TS11 and the brainwave signal for registration including a waveform W112 corresponding tothe induction sound TS12 are measured, and the authentication processand the registration process are started.

In the authentication process, matching is performed as to whether thewaveform W111 of the brain wave signal for authentication (input brainwave signal) has a similarity within a predetermined threshold valuerange with the waveform of the brain wave signal (registered brain wavesignal) recorded in the previous registration process, and in a casewhere there is a matched brain wave signal, the user is authenticated asa valid user. In the registration process, the brain wave signal forregistration including the waveform W112 is recorded for use in the nextauthentication process.

When the authentication using the induction sound is completed, a sound(authentication clear sound) or the like indicating that theauthentication is completed is output from the earphone 21A to the user(S212).

FIG. 27 illustrates an authentication registration example of the(N+1)-th electroencephalogram personal authentication.

When an induction sound is output from the earphone 21A and the userresponds to the auditory induction, a brain wave signal corresponding tothe response is measured. A combination of the (N+1)-th induction soundsis different from the combination of the above-described N-th inductionsounds (S213).

That is, the first half induction sound TS12 is an induction sound inwhich a brain wave signal is registered last time (N-th time), and thecurrent ((N+1)-th) authentication is performed using the brain wavesignal corresponding to the response of the user to the induction sound.The second half induction sound TS13 is an induction sound that is newlyand randomly generated, and a brain wave signal corresponding to aresponse of the user to the induction sound is registered for use in thenext ((N+2)-th) authentication.

In this example, the brain wave signal for authentication including thewaveform W112 corresponding to the induction sound TS12 and the brainwave signal for registration including a waveform W113 corresponding tothe induction sound TS13 are measured, and the authentication processand the registration process are started.

In the authentication process, matching is performed as to whether thewaveform W112 of the brain wave signal for authentication (input brainwave signal) has a similarity within a predetermined threshold valuerange with the waveform of the brain wave signal (registered brain wavesignal) recorded in the previous registration process, and in a casewhere there is a matched brain wave signal, the user is authenticated asa valid user. In the registration process, the brain wave signal forregistration including the waveform W113 is recorded for use in the nextauthentication process.

When the authentication using the induction sound is completed, a sound(authentication clear sound) or the like indicating that theauthentication is completed is output from the earphone 21A to the user(S214).

In this way, it is possible to perform authentication with furtherenhanced security by continuously performing authentication using thebrain wave signal corresponding to the auditory induction response bythe induction sound by changing the combination of the induction soundssuch as by performing authentication using the registered brain wavesignal registered at the time of the previous ((N−1)-th) authenticationand the input brain wave signal measured at the time of the current(N-th) authentication, and further performing authentication using theregistered brain wave signal registered at the time of the current(N-th) authentication and the input brain wave signal measured at thetime of the next ((N+1)-th) authentication.

Second Example

FIGS. 28 and 29 illustrate an authentication registration example ofelectroencephalogram personal authentication using visual induction.

In FIGS. 28 and 29 , the user wears on the head an HMD 21B connected toa terminal device 11 such as a game machine in a wireless or wiredmanner. The HMD 21B is provided with a plurality of electrodes, and canmeasure a brain wave signal from the head of the user. Note that anfNIRS signal may be measured instead of the brain wave signal.

FIG. 28 illustrates an authentication registration example of the N-thelectroencephalogram personal authentication.

When an induction video is displayed on the display of the HMD 21B andthe user responds to visual induction by the induction video, a brainwave signal corresponding to the response is measured (S221).

At this time, different videos are displayed in the first half and thesecond half of the induction video. That is, the induction video is acombination of the registered video and the new video, a differentcombination of videos is replayed each time.

The first half induction video TI21 is an induction video in which abrain wave signal is registered last time ((N−1)-th time), and thecurrent (N-th) authentication is performed using the brain wave signalcorresponding to the response of the user to the induction video. Thesecond half induction video TI22 is an induction video newly andrandomly generated, and a brain wave signal corresponding to a responseof the user to the induction video is registered for use in the next((N+1)-th) authentication.

In this example, the brain wave signal for authentication including awaveform W121 corresponding to the induction video TI21 and the brainwave signal for registration including a waveform W122 corresponding tothe induction video TI22 are measured, and the authentication processand the registration process are started.

In the authentication process, matching is performed as to whether thewaveform W121 of the brain wave signal for authentication (input brainwave signal) has a similarity within a predetermined threshold valuerange with the waveform of the brain wave signal (registered brain wavesignal) recorded in the previous authentication process, and in a casewhere there is a matched brain wave signal, the user is authenticated asa valid user. In the registration process, the brain wave signal forregistration including the waveform W122 is recorded for use in the nextauthentication process.

When the authentication by the induction video is completed, a video(authentication clear video) or the like indicating that theauthentication is completed are displayed on the display of the HMD 21Bfor the user (S222).

FIG. 29 illustrates an authentication registration example of the(N+1)-th electroencephalogram personal authentication.

When an induction video is displayed on the display of the HMD 21B andthe user responds to this visual induction, a brain wave signalcorresponding to the response is measured, but a combination of the(N+1)-th induction videos is different from the above-describedcombination of the N-th induction videos (S223).

That is, the first half induction video T122 is an induction video inwhich the brain wave signal is registered last time (N-th time), and thecurrent ((N+1)-th) authentication is performed using the brain wavesignal corresponding to the response of the user to the induction video.The second half induction video T123 is an induction video newly andrandomly generated, and a brain wave signal corresponding to a responseof the user to the induction video is registered for use in the next((N+2)-th) authentication.

In this example, the brain wave signal for authentication including thewaveform W122 corresponding to the induction video TI22 and the brainwave signal for registration including a waveform W123 corresponding tothe induction video TI23 are measured, and the authentication processand the registration process are started.

In the authentication process, matching is performed as to whether thewaveform W122 of the brain wave signal for authentication (input brainwave signal) has a similarity within a predetermined threshold valuerange with the waveform of the brain wave signal (registered brain wavesignal) recorded in the previous registration process, and in a casewhere there is a matched brain wave signal, the user is authenticated asa valid user. In the registration process, the brain wave signal forregistration including the waveform W123 is recorded for use in the nextauthentication process.

When the authentication by the induction video is completed, a video(authentication clear video) indicating that the authentication iscompleted is displayed on the display of the HMD 21B for the user forthe user (S224).

In this way, it is possible to perform authentication with furtherenhanced security by continuously performing authentication using thebrain wave signal corresponding to the visual induction response by theinduction video by changing the combination of the induction videos suchas by performing authentication using the registered brain wave signalregistered at the time of the previous ((N−1)-th) authentication and theinput brain wave signal measured at the time of the current (N-th)authentication, and further performing authentication using theregistered brain wave signal registered at the time of the current(N-th) authentication and the input brain wave signal measured at thetime of the next ((N+1)-th) authentication.

Note that, in the description of FIGS. 26 and 27 and FIGS. 28 and 29 ,the example using the brain wave signal is described, but, as describedabove, an fNIRS signal may be used. In this case, the “brain wavesignal” described above may be replaced with the “fNIRS signal”. Thebrain wave signal and the fNIRS signal are examples of braininformation. In addition, it is possible to perform theelectroencephalogram personal authentication for eachelectroencephalogram application.

Application Example of Content

By inserting an induction medium such as an induction sound or aninduction video into a content provided by a music distribution service,a game, or the like, continuous personal authentication and registrationcan be performed.

FIG. 30 illustrates an application example of a music contentdistributed by a music distribution service. In FIG. 30 , screens of amusic application displayed on the display of the mobile terminal 11Asuch as a smartphone or a mobile music player are illustrated in timeseries.

In FIG. 30 , it is possible to perform authentication using the brainwave signal according to the response of the user by the auditoryinduction by including the induction sound between the music and themusic or in the operation sound according to the operation of the userin a case where the user wearing the earphone 21A is listening to themusic content replayed by the mobile terminal 11A. Furthermore, theinduction sound may be inserted at the time of selecting, purchasing,downloading, or the like of the music content.

In this example, as a combination of the N-th induction sounds replayedbetween the pieces of music, the first half induction sound TS31 is aninduction sound registered last time ((N−1)-th time), and the current(N-th) authentication is performed using the brain wave signalcorresponding to the response of the induction sound. The second halfinduction sound TS32 is a newly generated induction sound, and a brainwave signal corresponding to a response of the induction sound isregistered for use in the next ((N+1)-th) authentication.

Thereafter, as a combination of the (N+1)-th induction sounds includedin the operation sound according to the user's operation, the first halfinduction sound TS32 is an induction sound registered last time (N-thtime), and the current ((N+1)-th) authentication is performed using thebrain wave signal according to the response of the induction sound. Thesecond half induction sound TS33 is a newly generated induction sound,and a brain wave signal corresponding to a response of the inductionsound is registered for use in the next ((N+2)-th) authentication.

As described above, during the replay of the music content, theinduction sound is replayed with the expression that fits the musiccontent and does not give a feeling of strangeness, so that the user cancontinuously perform the personal authentication and the registration ofthe authentication brain wave without being conscious. As a result, forexample, it is possible for the provider of the music distributionservice to appropriately authenticate the user who is viewing the musiccontent.

FIG. 31 illustrates an application example of the game content. In FIG.31 , game screens displayed on the display of the HMD 21B areillustrated in time series.

In FIG. 31 , it is possible to perform the authentication using thebrain wave signal corresponding to the user's response by the visualinduction by including the induction video in a screen on which a sceneof a game screen is being switched, and in a loading screen at the timeof data reading or the like in a case where the user wearing the HMD 21Bon the head is viewing the game content displayed on the display of theHMD 21B.

In this example, as a combination of the N-th induction videos includedin a certain loading screen, the first half induction video TI41 is aninduction video registered last time ((N−1)-th time), and the current(N-th) authentication is performed using the brain wave signalcorresponding to the response of the induction video. The second halfinduction video TI42 is a newly generated induction video, and a brainwave signal corresponding to a response of the induction video isregistered for use in the next ((N+1)-th) authentication.

Thereafter, as a combination of the (N+1)-th induction videos includedin the next and subsequent loading screens, the first half inductionvideo TI42 is an induction video registered last time (N-th time), andthe current ((N+1)-th) authentication is performed using the brain wavesignal corresponding to the response of the induction video. The secondhalf induction video 1143 is a newly generated induction video, and abrain wave signal corresponding to a response of the induction video isregistered for use in the next ((N+2)-th) authentication.

As described above, during the play of the game content, the inductionvideo is displayed with the expression that fits the game content anddoes not give a feeling of strangeness, so that the user cancontinuously perform the personal authentication and the registration ofthe authentication brain wave without being conscious. As a result, forexample, it is possible for the provider of the game content toappropriately perform authentication for the user playing the gamecontent.

In a screen presented in a game content, an online service, or the like,a loading screen, a user selection screen, or the like is often includedin the content. By reproducing the induction medium such as theinduction video as the background of the function display and options ofthese screens, the processes of the electroencephalogram personalauthentication and registration can be performed without the user'sconsciousness. Application examples of such a screen are illustrated inFIGS. 32 to 34 .

FIG. 32 illustrates an application example of the loading screen. FIG.32 illustrates an example in which in a case where a loading screen isdisplayed in the game content or the online service, the screenincluding the induction video TI51 is displayed. In this example, theinduction video TI51 is displayed as the background of the textindicating the progress of loading and the progress bar.

FIG. 33 illustrates an application example of the user selection screen.FIG. 33 illustrates an example in which in a case where a user selectionscreen is displayed in the game content or the online service, thescreen including the induction video TI61 is displayed. In this example,the induction video TI61 is displayed as the background of theselectable icons corresponding to the three users.

FIG. 34 illustrates an application example of the user selection screen.FIG. 34 illustrates an example in which in a case where a user selectionscreen is displayed in the game content, the screen including theinduction video TI71 is displayed. In this example, the induction videoTI71 is displayed as the background of the selectable characterscorresponding to the three users.

Note that, in the above description, the HMD 21B has been exemplified asthe device that executes the game content, but the induction video maybe displayed on a screen of a television receiver connected to the gamemachine, a screen of a portable game machine, or a screen of a displaydevice. The game content can include various contents such as, forexample, a virtual reality (VR) content and an augmented reality (AR)content in addition to a 2D content and a 3D content.

As described above, the induction medium is information for inducing andmeasuring the user's brain information according to the replay. Theinduction medium includes an induction sound, an induction video, or thelike, and is provided together with, for example, a distribution content(music content, game content, or the like) distributed by a contentdistribution service (music distribution service, Internet radio,podcast, game distribution service, and the like).

This providing method may include distributing the data of the inductionsound or the induction video as data of one type of content constitutingthe distribution content (distributing data of a music content or a gamecontent as another type), or inserting the data into data of the replaycontent constituting the distribution content (inserting the databetween music and music, an operation sound according to user'soperation, or the like). However, in the case of providing it as the onetype of content, the data of the induction sound or the induction videomay be generated by the device such as the mobile terminal 11A or theHMD 21B. Alternatively, the data of the induction sound or the inductionvideo may be added (for example, added in a superimposed manner to thebeginning or end of the music, the video or music being played in thegame.) by processing the data of the replay content constituting thedistribution content.

(System Configuration)

FIG. 35 illustrates a configuration example of a brain informationsystem to which the present technology is applied.

The brain information system is a system capable of providing variousservices using a brain wave signal measured from the head of the user.In FIG. 35 , the brain information system includes the terminal device11 and a measurement device 21. The same reference numerals are given toportions corresponding to those of the configuration example in FIG. 13, and the description thereof will be appropriately omitted.

The terminal device 11 is an electronic apparatus such as theabove-described mobile terminal 11A such as a smartphone or a mobilemusic player. As in the terminal device 10 in FIG. 13 , the terminaldevice 11 includes the control unit 101 and the communication unit 102.

The measurement device 21 is an electronic apparatus such as theearphone 21A and the HMD 21B described above. As in the measurementdevice 20 of FIG. 13 , the measurement device 21 includes the electrodes201-1 to 201-n, the reference electrode 202, the sensor unit 203, thesound output unit 204, the display unit 205, the input signal processingunit 206, the signal recording unit 207, the communication unit 208, theoutput processing unit 209, and the battery 210.

The input signal processing unit 206 processes the biometric signalsfrom the electrodes 201-1 to 201-n and the reference electrode 202,reads the brain wave signal, and performs a predetermined signalprocess.

Another Configuration Example

FIG. 36 illustrates another configuration example of the braininformation system to which the present technology is applied.

In FIG. 36 , the authentication system includes the terminal device 11and a measurement device 22. The same components as those in theconfiguration example in FIG. 35 are denoted by the same referencenumerals, and the description thereof will be omitted because it isrepeated.

In FIG. 36 , the measurement device 22 is an electronic apparatus suchas an earphone or an HMD. The measurement device 22 is different fromthe measurement device 21 of FIG. 35 in that instead of the electrodes201-1 to 201-n, the reference electrode 202, and the input signalprocessing unit 206, fNIRS sensors 251-1 to 251-m (m: an integer of oneor more) and an fNIRS signal processing unit 252 are included.

The fNIRS sensors 251-1 to 251-m are attached to parts such as theuser's head and ear. The measured fNIRS signals of the fNIRS sensors251-1 to 251-m are supplied to the fNIRS signal processing unit 252. ThefNIRS signal processing unit 252 performs a predetermined signal processon the fNIRS signals from the fNIRS sensors 251-1 to 251-m.

FIG. 37 illustrates a configuration example of the fNIRS sensor 251. InFIG. 37 , the fNIRS sensor 251 includes a light transmission unit 261and a light reception unit 262.

The light transmission unit 261 and the light reception unit 262 areattached so as to be in close contact with the scalp of the user, andthe near-infrared light emitted by the light transmission unit 261 istransmitted through the skin tissue and received by the light receptionunit 262. In this way, the fNIRS signal is measured by using ameasurement method applying the principle of optical functional imagingin which the brain function is non-invasively mapped from the scalpusing near-infrared light.

Arrangement Example of fNIRS Sensors

FIG. 38 illustrates an arrangement example of the fNIRS sensor 251provided in an earphone 22A.

As illustrated in FIG. 38 , four fNIRS sensors 251-1 to 251-4 aredisposed at substantially equal intervals on the same circumferencearound the portion that outputs a sound on the face in contact with theear of the user, and the fNIRS signal can be measured.

FIG. 39 illustrates an arrangement example of the fNIRS sensor 251provided in the HMD 22B.

As illustrated in FIG. 39 , in the HMD 22B, eight fNIRS sensors 251-2 to251-9 are disposed in a straight line at substantially equal intervalson the face in contact with the forehead of the user, and the fNIRSsensor 251-1 is disposed, away from the eight sensors, at apredetermined position on the face in contact with the back of the head,and the fNIRS signal can be measured.

Note that, in the perspective view of FIG. 15 , among the fNIRS sensor251-1 to 251-9, the three fNIRS sensors 251-7 to 251-9 are notillustrated because they are in blind spots. In addition, forconvenience of explanation, the fNIRS sensor 251 is illustrated toappear in the appearance of the HMD 22B, but is actually provided on theface in contact with the forehead of the user, that is, at a positionnot visible in the perspective view of FIG. 39 .

Configuration Example of Table

FIG. 40 illustrates a configuration example of a table recorded in thesignal recording unit 207 of FIG. 35 or 36 .

The signal recording unit 207 records a CBPA table 234. The CBPA table234 is a table in which registration information including inductionmedia such as an induction sound and an induction video and data such asbrain information such as brain wave information (brain wave signal) andcerebral blood flow information (fNIRS signal and the like) is recordedin time series. Continuous brainwave personal authentication (CBPA)means electroencephalogram personal authentication that enhancessecurity while changing the induction medium by continuous use.

As illustrated in FIG. 41 , in the CBPA table 234, registrationinformation including the induction sound #1 and the brain wave signal#31, registration information including the induction sound #2 and thebrain wave signal #32, registration information including the inductionsound #3 and the brain wave signal #33, registration informationincluding the induction sound #4 and the brain wave signal #34,registration information including the induction sound #5 and the brainwave signal #35, registration information including the induction sound#6 and the brain wave signal #36, and the like are recorded in timeseries. In the authentication process, the induction sound registeredlast time and the response brain wave signal thereof are read from theCBPA table 234, and authentication is performed. In the registrationprocess, a newly generated induction sound and a response brain wavesignal thereof are newly registered in the CBPA table 234.

Note that, although not illustrated because the description is repeated,as in the electroencephalogram system illustrated in FIGS. 20 to 22described above, the brain information system illustrated in FIG. 35 or36 may adopt various configurations such as a configuration in which theserver 30 is provided and a configuration in which the terminal device10 is omitted.

(Flow of Processing)

Next, the authentication registration process of electroencephalogrampersonal authentication will be described with reference to theflowcharts of FIGS. 42 and 43 .

In this authentication registration process, after receiving a commandto start the authentication process from the CBPA implementationapplication executed by the terminal device 11 or the measurement device21 (“Yes” in S311), the process proceeds to step S312.

In step S312, the input signal processing unit 206 refers to the CBPAtable 234 of the signal recording unit 207 to determine whether there isa registered brain wave signal for personal authentication of the CBPAimplementation application.

In the determination process of step S312, when it is recognized thatthere is no registered brain wave signal, that is, it is the first timedue to the first confirmation, the process proceeds to step S313. Instep S313, the output processing unit 209 reproduces the newregistration induction medium to output the medium from the sound outputunit 204 or the display unit 205.

In step S314, the input signal processing unit 206 processes thebiometric signal from the electrode 201 and the reference electrode 202,and determines whether the brain wave signal according to the responseby the induction medium has been read.

In a case where it is determined in the determination process of stepS314 that the response brain wave signal has not been read, the processproceeds to step S315. In step S315, notification of an error isprovided to the CBPA implementation application executed by the terminaldevice 11 or the measurement device 21.

Note that, in a case where retry is defined by the CBPA implementationapplication, the process returns to step S313, and the subsequentprocesses are repeated, and reading of the response brain wave signalaccording to the new registration induction medium is retried.

On the other hand, in a case where it is determined in the determinationprocess of step S314 that the response brain wave signal has been read,the process proceeds to step S316. In step S316, the input signalprocessing unit 206 records information regarding the induction medium,the response brain wave signal, and the time in the CBPA table 234 ofthe signal recording unit 207.

When the process of step S316 is completed, the process proceeds to stepS317. In addition, in a case where it is determined in the determinationprocess of step S312 that there is the registered brain wave signal, theprocess proceeds to step S317.

In step S317, the output processing unit 209 reproduces the inductionmedium registered last time to output the medium from the sound outputunit 204 or the display unit 205.

In step S318, the input signal processing unit 206 processes thebiometric signal from the electrode 201 and the reference electrode 202,and determines whether the brain wave signal according to the responseby the induction medium has been read.

In a case where it is determined in the determination process of stepS318 that the response brain wave signal has not been read, the processproceeds to step S319. In step S319, notification of an error isprovided to the CBPA implementation application executed by the terminaldevice 11 or the measurement device 21.

Note that, in a case where retry is defined by the CBPA implementationapplication, the process returns to step S317, and the subsequentprocesses are repeated, and reading of the response brain wave signalaccording to the induction medium registered last time is retried.

On the other hand, in a case where it is determined in the determinationprocess of step S318 that the response brain wave signal has been read,the process proceeds to step S320 of FIG. 42 .

In step S320, the input signal processing unit 206 performs matchingbetween the input brain wave signal read by the process in step S318 andthe registered brain wave signal recorded in the CBPA table 234 of thesignal recording unit 207. In addition, in step S321, the input signalprocessing unit 206 calculates the similarity between the matched brainwave signals.

In step S322, the input signal processing unit 206 determines whetherthe calculated similarity is within a predetermined threshold valuerange.

In a case where it is determined in the determination process of stepS322 that the similarity is out of the range of the predeterminedthreshold value, the process proceeds to step S323. In step S323,notification of an error is provided to the CBPA implementationapplication executed by the terminal device 11 or the measurement device21, and the authentication registration process ends.

In addition, in a case where it is determined in the determinationprocess of step S322 that the similarity falls within the range of thepredetermined threshold value, the process proceeds to step S324. Instep S324, the output processing unit 209 reproduces the newregistration induction medium to output the medium from the sound outputunit 204 or the display unit 205.

In step S325, the input signal processing unit 206 processes thebiometric signal from the electrode 201 and the reference electrode 202,and determines whether the brain wave signal according to the responseby the induction medium has been read.

In a case where it is determined in the determination process of stepS325 that the response brain wave signal has not been read, the processproceeds to step S326. In step S326, an error is notified to the CBPAimplementation application executed by the terminal device 11 or themeasurement device 21.

Note that, in a case where retry is defined by the CBPA implementationapplication, the process returns to step S324, and the subsequentprocesses are repeated, and reading of the response brain wave signalaccording to the new registration induction medium is retried.

On the other hand, in a case where it is determined in the determinationprocess of step S325 that the response brain wave signal has been read,the process proceeds to step S327. In step S327, the input signalprocessing unit 206 records information regarding the induction medium,the response brain wave signal, and the time in the CBPA table 234 ofthe signal recording unit 207.

When the process of step S327 is completed, the process proceeds to stepS328. In step S328, it is determined whether to end the action, and in acase where it is determined that the action is not to be ended, theprocess returns to step S311 in FIG. 42 , and the subsequent processesare repeated. In addition, in a case where it is determined that theaction is to be ended, the authentication registration process ends.

As described above, the measurement device 21 or the measurement device22 is continuously used to repeat authentication and registration usingbrain information such as a brain wave signal or an fNIRS signalaccording to a response of the user to the replayed induction mediumwhile changing the induction medium such as auditory induction or visualinduction, thereby enhancing security.

That is, in the present technology, control of a first authenticationprocess of authenticating a user on the basis of first registrationinformation based on first brain information about the user measured inresponse to a replay of a first induction medium, and control of aregistration process of a second registration information used for asecond authentication process of authenticating the user on the basis ofsecond brain information about the user measured in response to a replayof a second induction medium are performed. Therefore, since theauthentication process and the registration process are repeated whilechanging the induction medium, it is possible to suppress thepossibility that the induction medium or the brain information leaks tothe outside due to hacking or the like, and perform authentication withhigher safety using the brain information.

Note that, in the above description, the induction sound as the auditoryinduction and the induction video as the visual induction have beenexemplified as the induction medium, but an induction vibration as thetactile induction, the olfactory induction, or the like may be used. Inaddition, in each authentication, a replay is performed with acombination of the first induction medium and the second inductionmedium, and a combination of the first induction medium and the secondinduction medium replayed in each authentication is different. Forexample, the replay order of the first induction medium (forauthentication) and the second induction medium (for registration) isnot limited to the order of the first induction medium and the secondinduction medium, and may be the order of the second induction mediumand the first induction medium, or the order may be changed each time inother than continuing a certain order.

3. Configuration of Computer

A series of processes of the terminal device 10, the measurement device20, and the server 30 described above can be executed by hardware orsoftware. In a case where the series of processes is executed bysoftware, a program constituting the software is installed in a computerof each device.

FIG. 44 is a block diagram illustrating a configuration example ofhardware of a computer that executes the above-described series ofprocesses by a program.

In a computer, a central processing unit (CPU) 1001, a read only memory(ROM) 1002, and a random access memory (RAM) 1003 are mutually connectedby a bus 1004. An input/output interface 1005 is further connected tothe bus 1004. An input unit 1006, an output unit 1007, a storage unit1008, a communication unit 1009, and a drive 1010 are connected to theinput/output interface 1005.

The input unit 1006 includes a microphone, a keyboard, a mouse, and thelike. The output unit 1007 includes a speaker, a display, and the like.The storage unit 1008 includes a hard disk, a nonvolatile memory, andthe like. The communication unit 1009 includes a network interface andthe like. The drive 1010 drives a removable recording medium 1011 suchas a magnetic disk, an optical disk, a magneto-optical disk, or asemiconductor memory.

In the computer configured as described above, the CPU 1001 loads aprogram recorded in the ROM 1002 or the storage unit 1008 into the RAM1003 via the input/output interface 1005 and the bus 1004 and executesthe program, whereby the above-described series of processes isperformed.

The program executed by the computer (CPU 1001) can be provided by beingrecorded in the removable recording medium 1011 as a package medium orthe like, for example. In addition, the program can be provided via awired or wireless transmission medium such as a local area network, theInternet, or digital satellite broadcasting.

In the computer, the program can be installed in the storage unit 1008via the input/output interface 1005 by attaching the removable recordingmedium 1011 to the drive 1010. In addition, the program can be receivedby the communication unit 1009 via a wired or wireless transmissionmedium and installed in the storage unit 1008. In addition, the programcan be installed in the ROM 1002 or the storage unit 1008 in advance.

Here, in the present specification, the processing performed by thecomputer according to the program is not necessarily performed in timeseries in the order described as the flowchart. That is, the processingperformed by the computer according to the program also includesprocessing executed in parallel or individually (for example, parallelprocessing or processing by an object).

In addition, the program may be processed by one computer (processor) ormay be processed in a distributed manner by a plurality of computers.Further, the program may be transferred to a remote computer andexecuted.

Furthermore, in the present specification, a system means a set of aplurality of components (devices, modules (parts), etc.), and it doesnot matter whether or not all the components are in the same housing.Therefore, a plurality of devices housed in separate housings andconnected via a network and one device in which a plurality of modulesis housed in one housing are both systems.

Note that the embodiments of the present technology are not limited tothe above-described embodiments, and various modifications can be madewithout departing from the gist of the present technology. For example,the present technology can have a configuration of cloud computing inwhich one function is shared and processed in cooperation by a pluralityof devices via a network.

In addition, each step described in the above-described flowchart can beexecuted by one device or can be shared and executed by a plurality ofdevices. Furthermore, in a case where a plurality of processes isincluded in one step, the plurality of processes included in the onestep can be executed by one device or can be shared and executed by aplurality of devices.

In addition, the effects described in the present specification aremerely examples and are not limited, and other effects may be provided.

Note that the present technology can have the following configurations.

(1)

A control device including

a detection unit configured to perform detection of a brain waveincluded in a measured biometric signal of a user and detection of auser action based on information other than the brain wave included inthe biometric signal, and

a processing unit configured to perform a predetermined process based onthe brain wave in a case where the user action is a predeterminedaction.

(2)

The control device according to the item (1), in which

the information other than the brain wave includes a myoelectric signal,and

the detection unit detects the user action on the basis of themyoelectric signal measured from the user.

(3)

The control device according to the item (2), in which

the predetermined action is a trigger action set according to anapplication using a brain wave.

(4)

The control device according to the item (3), in which

in a case where a trigger action is detected as the user action, theprocessing unit performs a matching process between a first brain wavesignal measured from the user and a second brain wave signal measuredfrom the user at timing different from timing of the first brain wavesignal.

(5)

The control device according to the item (2), in which

the predetermined action is a trigger action set for each applicationusing a brain wave.

(6)

The control device according to the item (2) or (5), in which

the application is a personal authentication application using a brainwave, and

in a case where a trigger action is detected as the user action, theprocessing unit performs a matching process between a first brain wavesignal measured from the user and a second brain wave signal measuredfrom the user at timing different from timing of the first brain wavesignal.

(7)

The control device according to any one of the items (4) to (6), inwhich

the first brain wave signal and the second brain wave signal aremeasured according to an instruction.

(8)

The control device according to any one of (4) to (6), in which

the first brain wave signal and the second brain wave signal areevent-related potentials.

(9)

The control device according to the item (2) or (5), in which

the application is a search application using a brain wave, and

a first brain wave signal measured from the user and a second brain wavesignal measured from the user at timing different from timing of thefirst brain wave signal each include a brain wave signal regarding athought of the user with respect to a same object.

(10)

The control device according to the item (9), in which

the object includes information allowed to be presented to a terminaldevice used by the user, and provided by a specific medium.

(11)

The control device according to any one of the items (3) to (10), inwhich

the processing unit

performs a matching process between a first myoelectric signal measuredfrom the user and a second myoelectric signal measured from the user attiming different from timing of the first myoelectric signal, and

determines whether a trigger action is detected on the basis of a resultof the matching process.

(12)

The control device according to the item (11), in which

the first myoelectric signal is measured and registered in advance foreach application, and

the second myoelectric signal is measured at any timing when anapplication is activated.

(13)

The control device according to any one of the items (3) to (12), inwhich

the trigger action includes a movement of each part of a head of theuser or a combination of the movement of the each part and a movement ofthe head.

(14)

The control device according to the item (13), in which

the trigger action is detected by a signal from an electrode provided tocontact a head of the user or a signal from a sensor unit.

(15)

The control device according to the item (4), in which

the brain wave signal and the myoelectric signal are each measured by asignal from an electrode provided to contact a head of the user.

(16)

The control device according to any one of the items (2) to (15), inwhich

the control device has a first mode in which a myoelectric signal ismeasured and a second mode in which a brain wave signal is measured, and

sets a time resolution and an amplification factor of a signal amplifierat a time of reading a signal in the first mode to be lower than a timeresolution and an amplification factor at a time of reading a signal inthe second mode.

(17)

The control device according to any one of the items (2) to (16), inwhich

the control device is configured as a measurement device that measures abrain wave signal and a myoelectric signal, a terminal device connectedto the measurement device in a wired or wireless manner, or a serverconnected to the measurement device via a network.

(18)

A control method including

a control device

detecting a brain wave included in a measured biometric signal of a userand detecting a user action based on information other than the brainwave included in the biometric signal, and

performing a predetermined process based on the brain wave in a casewhere the user action is a predetermined action.

REFERENCE SIGNS LIST

-   10, 11 Terminal device-   10A, 11A Mobile terminal-   20, 21, 22 Measurement device-   20A, 21A Earphone-   20B, 21B HMD-   30 Server-   40 Network-   101 Control unit-   102 Communication unit-   103 Input signal processing unit-   104 Signal recording unit-   105 Output processing unit-   106 Sound output unit-   107 Display unit-   201, 201-1 to 201-n Electrode-   202 Reference electrode-   203 Sensor unit-   204 Sound output unit-   205 Display unit-   206 Input signal processing unit-   207 Signal recording unit-   208 Communication unit-   209 Output processing unit-   210 Battery-   211 Control unit-   221 Acceleration sensor-   222 Gyro sensor-   231 Trigger action signal recording table-   232 Trigger action-specific electroencephalogram application table-   233 Brain wave signal recording table-   234 CBPA table-   251, 251-1 to 251-m fNIRS sensor-   252 fNIRS signal processing unit-   261 Light transmission unit-   262 Light reception unit-   301 Control unit-   302 Communication unit-   303 Input signal processing unit-   304 Signal recording unit-   305 Output processing unit-   1001 CPU

1. A control device comprising: a detection unit configured to performdetection of a brain wave included in a measured biometric signal of auser and detection of a user action based on information other than thebrain wave included in the biometric signal; and a processing unitconfigured to perform a predetermined process based on the brain wave ina case where the user action is a predetermined action.
 2. The controldevice according to claim 1, wherein the information other than thebrain wave includes a myoelectric signal, and the detection unit detectsthe user action on a basis of the myoelectric signal measured from theuser.
 3. The control device according to claim 2, wherein thepredetermined action is a trigger action set according to an applicationusing a brain wave.
 4. The control device according to claim 3, whereinin a case where a trigger action is detected as the user action, theprocessing unit performs a matching process between a first brain wavesignal measured from the user and a second brain wave signal measuredfrom the user at timing different from timing of the first brain wavesignal.
 5. The control device according to claim 2, wherein thepredetermined action is a trigger action set for each application usinga brain wave.
 6. The control device according to claim 5, wherein theapplication is a personal authentication application using a brain wave,and in a case where a trigger action is detected as the user action, theprocessing unit performs a matching process between a first brain wavesignal measured from the user and a second brain wave signal measuredfrom the user at timing different from timing of the first brain wavesignal.
 7. The control device according to claim 6, wherein the firstbrain wave signal and the second brain wave signal are measuredaccording to an instruction.
 8. The control device according to claim 6,wherein the first brain wave signal and the second brain wave signal areevent-related potentials.
 9. The control device according to claim 5,wherein the application is a search application using a brain wave, anda first brain wave signal measured from the user and a second brain wavesignal measured from the user at timing different from timing of thefirst brain wave signal each include a brain wave signal regarding athought of the user with respect to a same object.
 10. The controldevice according to claim 9, wherein the object includes informationallowed to be presented to a terminal device used by the user, andprovided by a specific medium.
 11. The control device according to claim3, wherein the processing unit performs a matching process between afirst myoelectric signal measured from the user and a second myoelectricsignal measured from the user at timing different from timing of thefirst myoelectric signal, and determines whether a trigger action isdetected on a basis of a result of the matching process.
 12. The controldevice according to claim 11, wherein the first myoelectric signal ismeasured and registered in advance for each application, and the secondmyoelectric signal is measured at any timing when an application isactivated.
 13. The control device according to claim 3, wherein thetrigger action includes a movement of each part of a head of the user ora combination of the movement of the each part and a movement of thehead.
 14. The control device according to claim 13, wherein the triggeraction is detected by a signal from an electrode provided to contact ahead of the user or a signal from a sensor unit.
 15. The control deviceaccording to claim 4, wherein the brain wave signal and the myoelectricsignal are each measured by a signal from an electrode provided tocontact a head of the user.
 16. The control device according to claim 2,wherein the control device has a first mode in which a myoelectricsignal is measured and a second mode in which a brain wave signal ismeasured, and sets a time resolution and an amplification factor of asignal amplifier at a time of reading a signal in the first mode to belower than a time resolution and an amplification factor at a time ofreading a signal in the second mode.
 17. The control device according toclaim 2, wherein the control device is configured as a measurementdevice that measures a brain wave signal and a myoelectric signal, aterminal device connected to the measurement device in a wired orwireless manner, or a server connected to the measurement device via anetwork.
 18. A control method comprising: a control device detecting abrain wave included in a measured biometric signal of a user anddetecting a user action based on information other than the brain waveincluded in the biometric signal; and performing a predetermined processbased on the brain wave in a case where the user action is apredetermined action.